The present invention relates to image processing, and, more particularly, to an analog front end for a charge coupled device, which provides digital optical black and offset correction and noise filtering.
Great strides in integrated circuit design and manufacturing have enabled low cost, highly integrated, high performance image processing products, including the digital electronic cameras. A conventional camera comprises an image sensor, typically an array charge coupled device (CCD), an analog front end (AFE) and a digital image processor. Most analog front ends having optical black and offset calibration include schemes that integrate the error signal on a capacitor during an optical black period and feed back the voltage generated to the input to cancel the offset or the optical black value during the video interval.
As shown in circuit 100 of FIG. 1, the CCD 102, an integrated array of photocells used in digital imaging, is connected to a capacitor 104 and a clamp circuit 106 for AC coupling. The AFE connected to the capacitor 104 generally includes three main elements: a correlated double sampler 108 (CDS), a programmable gain amplifier 110 (PGA), and an analog to digital converter 112 (ADC). The fundamental goal in any camera design is to extract as much dynamic range from the image sensor without adding any noise with the subsequent circuitry.
The specific operation of the conventional image process apparatus 100 with such a construction is described referring to the timing charts of CCD 102 output in FIGS. 2a and 2b. Particularly, the output of the CCD 102 contains a reset pulse, the reference level and the video level. Output from the CCD 102 is sampled twice by CDS 108 such that the first sample is taken during the reference level and the second sample is taken during the video signal. The difference is the corresponding CDS 108 output. This difference between the optical black level and the video level represents the actual image value for any given pixel.
As shown in FIG. 2b, a dark cell does not produce a zero differential output, due to the dark currents of the photocells, which may vary from pixel to pixel and line to line in a frame. Due to the dark current or xe2x80x9coptical black levelxe2x80x9d and the internal offsets of all amplifiers used in the CDS 108, PGA 110 and ADC 112, the resulting ADC 112 output for a dark cell will not be zero. Further complicating the matter, the CDS 108 offset and the optical black level are multiplied by the gain from the PGA 110. In order to achieve the ideal dynamic range for the signal, the black level and the offsets must be removed.
The function of the CDS 108, as illustrated in FIGS. 2a and 2b, is to sense and produce a voltage difference between the reference level and the video level of each pixel. The most important benefit of using CDS 108 is to reduce noise. In addition to the capturing of the video data by subtracting the reference levels from the video levels, any noise common to these two signals are removed by the CDS 108.
One approach for canceling an offset in switched capacitor amplifiers is to put the amplifier in unity gain feedback during the sampling phase. This way the input offset is also sampled and canceled during the amplification phase. For applications, however, where high speed and high closed loop gain are required, stable amplifiers at unity gain feedback can not be maintained. In addition, this approach will not correct the optical black level.
Another approach corrects the optical black level using the feedback circuit 300 displayed in FIG. 3. It integrates the optical black error on an integrator and applies a negative feedback to the input of the PGA 306. The feedback circuit operates to control the level of the analog optical black signal to a predetermined level.
This technique, however, lacks the flexibility of digital programmability and requires analog circuit complexity, sometimes even off-chip capacitors. It is also not suitable for discrete time (switched capacitor) systems because of the latency at the amplifier outputs. In the alternative, however, post digital optical black correction techniques is not desired, since it is better to cancel the offset in analog domain for an optimum dynamic range.
Our copending application Ser. No. 09/353,919, as shown in FIG. 4, provides a CCD signal processing method that provides optical black offset correction using a moving average filter scheme such that the optical black pixels are averaged at the beginning of each line and offset DAC, 418, are updated in order to cancel the offset. The analog front end (AFE) converts the CCD output signal to digital data to allow subsequent digital signal processing. At the input of the AFE, the DC level of the CCD output signal is clamped to the input dynamic range. For better noise performance and dynamic range, correlated double sampling is applied to the clamped input signal. The output of correlated double sampler (CDS) is amplified by a programmable gain that varies exponentially with linear control. Then the amplified analog signal is converted to digital data. The optical black value and channel offset are corrected in order to maximize the dynamic range.
Using a feedback loop having a switch 410 that closes during optical black level sampling of the signal, a digital averager 412 averages the optical black pixels. A comparator 414 compares the desired optical black level with the averaged optical black level. It provides an up and down control signal to the up/down counter 416. Counter/register 416 counts up or down until the output of the ADC 408 converges to the desired optical black level. Digital-to-analog converter 418 converts the output of the counter into an analog voltage to be applied to the image signal output from CDS 402. This circuit arrangement, however, will take an unknown repetition of feedback lines to cancel the optical black level offset. Also, if the PGA gain is too high, the accuracy of the cancellation may be poor.
The second embodiment in our copending application Ser. No. 09/353,919, provides a CCD signal processing method that provides optical black offset correction using a moving average filter scheme such that the optical black pixels are averaged at the beginning of each line and offset DACs, DAC-C 612 and DAC-F 614, are updated in order to cancel the offset. Specifically, as shown in FIG. 6, circuit 600 includes a mixed signal technique that corrects the offset and optical black value in the analog domain using a coarse and fine adjustment mode. Digital optical black correction circuit 616 determines the necessary amount that the analog offset of the image signal should be adjusted. DAC-C 612 and DAC-F 614 provide offsets in the coarse and fine adjustment modes, respectively. This highly programmable technique can be used both in discrete and continuous time systems and does not require any off-chip components.
In operation, CCD image lines are shifted vertically to a line register, then the pixels on this line are shifted horizontally to an output pin. This process causes a gradual increase in the optical black value within the frame, which needs to be corrected. As shown in FIG. 7, there may be an initial jump in the optical black value for the first line of the image frame or field. This jump is caused by different exposure times. Afterwards there is a gradual increase in the average value. In addition to the slow ramp due to the shift in the optical black value during the image read mode, line noise exists as shown; thus, if correction DACs are updated every line, there will be line noise. If DAC updates are conducted over a fixed number of user programmable lines, then there may be visible bands on the image. Moreover, the average differs from line to line since some of the optical black pixels may be defective, i.e. hot and cold optical black pixels. A hot pixel is a defective pixel that generates too much charge, and a cold pixel is the one that does not generate any charge.
There exists a need for a moving average filter scheme for CCD optical black correction to remove line noise and hot and cold pixels without creating bands on the image, wherein straightforward moving average filter can be used, or a simplified version can be used in order to save significant amount of registers and complex digital circuits.
To address the above-discussed deficiencies of the analog front end circuitry having optical black and offset correction, the present invention teaches an offset and optical black correction circuit having a digital filter to obtain noise-free optical black correction for charge-coupled devices such that a digitally programmable bandwidth exists. In accordance with the present invention, the sum of the channel offset and optical black level present at the output of the ADC as a digital error signal with high frequency noise components passes through hot/cold pixel filtering and is averaged for a given number of lines and optical black cells per line. Finally, it passes through a digital filter with programmable bandwidth to generate a filtered digital error signal. This error signal is fed back to the analog channel through digital to analog conversion in order to obtain the desired optical black level at the output of the ADC.
A first embodiment of the image processing apparatus in accordance with the present invention includes an analog front end circuit connected between the CCD and the optical black and offset correction circuit. The apparatus includes a sampling circuit for sampling the incoming image signal and a detecting circuit for detecting the optical black level. A digital averager averages the optical black pixels at the beginning of each line of the image signal. In addition, the digital averager includes a first filter for filtering hot and cold pixels prior to averaging the optical black pixels. A line noise filter receives the averaged optical black signal and, further, removes line noise from the optical black signal. A digital comparator receives a reference signal and the optical black signal to compare the optical black signal with the reference signal. The difference is received by a correction circuit for correcting the optical black level by feeding back the difference obtained by the digital comparator such that the difference is applied to the analog image signal. Within the present embodiment, the correction circuit includes a digital-to-analog converter that converts the difference back to an analog signal to be applied as an adjustment to the analog image signal at the input of the image processing apparatus.
In a second embodiment, the correction circuit includes a first and a second digital-to-analog converter, used to apply a coarse and fine adjustment to the image signal at differing points in the image processing. As known, the AFE generally includes three main elements: a correlated double sampler (CDS), a programmable gain amplifier (PGA) and an analog-to-digital converter (ADC). The coarse adjustment would first be applied to the image signal before to the PGA and the fine adjustment would be applied to the amplified image signal after the PGA. This embodiment increases the accuracy when the gain of the PGA is high.
Advantages of this design include but are not limited to an analog front end circuit having mixed signal optical black and offset circuitry that is highly programmable which eliminates line noise and cold/hot pixels. This circuit has an improved dynamic range for image processing over other approaches. As such, this highly programmable design can be used both in discrete and continuous time systems and does not require any off-chip components. Thus, this design meets the goal of extracting as much analog dynamic range from the image sensor without adding any noise with the subsequent circuitry.